Learning Adversarial Markov Decision Processes with Delayed Feedback

Reinforcement learning typically assumes that agents observe feedback for their actions immediately, but in many real-world applications (like recommendation systems) is observed delay. This paper studies online episodic Markov decision processes (MDPs) with unknown transitions, adversarially changing costs and unrestricted delayed feedback. That is, the trajectory of episode k are revealed to learner only end k+d?, where delays d? neither identical nor bounded, chosen by an oblivious adversary. We present novel algorithms based on policy optimization achieve near-optimal high-probability regret (K+D)¹?² under full-information feedback, K number episodes D=?? total Under bandit we prove similar assuming stochastic, (K+D)²?³ general case. first consider minimization important setting MDPs